Methods and systems for very precisely positioning at least one object into a defined final position in space by means of an industrial robot and a measuring system are used, in particular, in production and assembly processes along automated assembly lines, for example in the automotive industry. Here an object, for example a sheet body part or any other body part, should, by means of an industrial robot, be brought, very precisely, into a specific position and alignment in space in order to carry out a work step.
The prior art has disclosed handling systems, more particularly industrial robots, e.g. articulated robots, for defined positioning of an object into a predetermined position and alignment in space, which object is gripped by means of a gripper device. Here, the industrial robots can have internal measuring systems that can capture the position of the limbs of the handling system and hence provide information in respect of the position and alignment of the gripper device in space.
Hence it is possible to move the gripper device, including the gripped object, into a specific predefined position by means of an appropriate input for the robot control. Hence, the gripped object is positioned in space by prescribing a position of the gripper device. However, it is the two following problems in particular that arise from this.
Firstly, the internal measuring system of conventional industrial robots designed for holding heavy objects is not precise enough for allowing the gripper device to assume a position in space that is that precise as required for some assembly methods. Although the drives of industrial robots are sufficiently precise, their measuring systems are not. The kinematic chain multiplies the measuring errors of the individual measuring members. This results from both the measurement inaccuracies of the individual measuring members, more particularly the angle measurers of an articulated robot, and the unavoidable elasticity of the robot members.
Secondly, the position of the gripper device, and hence the position thereof in space, does not necessarily provide the position of the object in space because the object can usually only be gripped within a gripping tolerance. This gripping tolerance is often far greater than the required positioning accuracy. Hence the gripping error, i.e. the relative position of the object with respect to the gripper device, must likewise be taken into account. To this end, use is made of separate measuring systems, more particularly contactless optical measuring systems, which are no longer part of the robot. Only these measuring systems allow the object to be positioned at a specific position in space with the required accuracy.
Such contactless measuring systems, which can be used for positioning an object very precisely into a final position in space within the scope of an industrial process, are for example described in the two European patent applications numbered 07124101.2 and 09161295.2, as outlined below.
The method described in the European patent application numbered 07124101.2 is carried out by means of an industrial robot, a first optical recording apparatus and at least a second optical recording apparatus. The first industrial robot can be moved into predetermined positions. It is calibrated internally, as well as calibrated in the three-dimensional coordinate system of the space and related to the latter. The first optical recording apparatus, which is calibrated in a three-dimensional coordinate system of the space and positioned at a known first position with a known alignment, comprises an optically calibrated first camera for recording an image within a specific first field of view; a first drive unit for aligning the first camera, which brings about a change in the first field of view; and a first angle measuring unit, calibrated in the coordinate system of the space, for very precisely capturing the angular alignment of the first camera such that the first field of view can be determined in the coordinate system of the space. The at least one second optical recording apparatus, which is calibrated in the three-dimensional coordinate system of the space and positioned at a known second position with a known alignment, comprises an optically calibrated second camera for recording an image within a specific second field of view; a second drive unit for aligning the second camera, which brings about a change in the second field of view; and a second angle measuring unit, calibrated in the coordinate system of the space, for very precisely capturing the angular alignment, of the second camera such that the second field of view can be determined in the coordinate system of the space. The at least two positions, i.e. the position of the first and the second recording apparatus, are spaced apart such that three-dimensional image recording of the at least one object by means of the at least two recording apparatuses is made possible as a result of at least partly overlapping fields of view.
The method comprises the following steps:
A first object, having known first features that can be captured by optical means, is gripped and held within a gripping tolerance by the first industrial robot.
Such a first compensating variable, which corrects the gripping tolerance, is determined for the first industrial, robot such that the first object can be moved in compensated fashion in the coordinate system of the space by prescribing a position of the first industrial robot. The first compensating variable is determined by the following steps: respectively using the drive units for aligning the at least two cameras, with at least partly overlapping fields of view of the cameras, with respect to at least some of the first features of the first object, which is held in a first compensating position of the first industrial robot; recording first image recordings by means of the two cameras; determining the position of the first object in the first compensating position of the first industrial robot in the coordinate system of the space by using the positions of the recording apparatuses, the angular alignments of the cameras that were captured by the angle measuring units, the first image recordings and knowledge in respect of the first features on the first object; and determining the first compensating variable by using the first compensating position of the first industrial robot and at least the determined position of the first object in the first compensating position of the first industrial robot.
By means of the following repeating steps, the first object is moved very precisely into a first final position until the first final, position is reached within a predetermined tolerance:
Recording further first image recordings using the cameras; determining the current position of the first object in the coordinate system of the space using the positions of the recording apparatuses, the angular alignments of the cameras captured by the angle measuring units, the additional first image recordings and knowledge in respect of the first features on the first object; calculating the difference in position between the current position of the first object and the first final position; calculating a new intended position of the first industrial robot taking into account the first compensating variable from the current position of the first industrial robot and a variable linked to the difference in position; and moving the first industrial robot into the new intended position.
Moreover, the European patent application numbered 07124101.2 describes the same system for very precisely positioning at least one object into a final position in space comprising an industrial robot, a first optical recording apparatus, a second optical recording apparatus and a control unit. Here, the control unit is used to control the recording apparatuses and the industrial robot such that these accordingly carry out the method described above.
This described method and corresponding system is particularly distinguished by flexibility, precision and fast process speed.
The European patent application numbered 09161295.2—in parallel to the method and system from the European patent application numbered 07124101.2—also describes such a method and system for very precisely positioning at least one object into a final position in space, wherein, however, 3D image recording apparatuses are utilized as recording apparatuses.
Here, she prior art has disclosed different 3D image recording apparatuses. By way of example, there are 3D image recording apparatuses that are substantially composed of two or three cameras, wherein she cameras are housed, fixedly coupled to one another, in a common housing with a distance between them—i.e. with a stereo basis—for recording a scene from respectively different perspectives, with however said perspectives being fixed relative to one another. Since the recorded area section does not necessarily have characteristic image features that make electronic processing of the images possible, markings may be applied to the area section. These markings can be generated by means of a structured beam of light, more particularly a laser beam, projected onto the area section by the 3D image recording unit, which for example projects an optical grid or an optical marking cross. Such 3D image recording units usually also contain an image processing apparatus, which derives a three-dimensional image from the plurality of images from different perspectives, which images were recorded substantially simultaneously.
By way of example, such 3D image recording units include the image recording systems made by “CogniTens”, which are marketed under the names of “Optigo” and “OptiCell” and contain three cameras arranged in an equilateral triangle, and also the system “Advent” from “ActiCM”, which has two high-resolution CCD cameras arranged next to one another and a projector for projecting structured light onto the section to be recorded.
The coordinates of recorded image elements to be measured are usually determined by means of referenced markings within the image, with these markings forming the basis for the actual 3D coordinate measurement. Herein, the image coordinate system, which relates to the recorded three-dimensional image and hence is related to the 3D image recording unit, is transformed into the object coordinate system, within which the object should be measured and which for example is based on the CAD model of the object. The transformation is undertaken on the basis of recorded reference markings, the positions of which are known in the object coordinate system. The 3D image recording units known from the prior art herein achieve accuracies of less than 0.5 millimeters.
Furthermore, 3D scanning systems are known, more particularly in the form of 3D scanners with electro-optical distance measurement; these carry out depth scanning within an area region and generate a point cloud. Here, a distinction should be made between serial systems, in which a point-like measurement beam scans an area point-by-point; parallel, systems, in which a line-like measurement beam scans an area line-by-line; and fully parallel systems, which simultaneously scan a multiplicity of points within an area region and hence carry out a depth recording of the area region. In general, what is common to all these systems is that the depth scanning is carried out by means of at least one distance measurement beam that is directed at the area and/or moved over the area.
Moreover, there are RIM cameras, also referred to as RIMs or range imaging systems, which can be used to record an image of an object while at the same time capturing depth information for each pixel or a group of pixels. Hence it is possible to use a single device for capturing a three-dimensional image, in which depth information, i.e. distance information from the camera, is assigned to each pixel or to a multiplicity of pixel groups.
WO 2007/004983 A1 (Pettersson) has disclosed a method for welding together workpieces, more particularly pressed sheet parts or composite sheets. The workpieces to be joined together are held by industrial robots and are positioned relative to one another by the latter for being joined together by welding. During the production of the welding joint, the workpieces are held in the respective positions by the industrial robots such that the relative position of the parts with respect to one another is maintained. By way of example, a welding robot undertakes the welding. A measuring system measures the positions of the workpieces in order to enable the workpieces to be positioned before the welding procedure. In particular, there is continuous measuring during the welding procedure. The described method affords the possibility of dispensing with the otherwise conventional workpiece-specific molds and workpiece receptacles, which are laborious to produce, and into which the workpieces have to be fixed prior to the welding. The industrial robots can be used universally for differently shaped and embodied workpieces because the process of capturing the position of the workpieces by means of the measuring system allows identification and monitoring of the workpieces, and also precise relative positioning of the parts with respect to one another. Hence a single system can be used for different workpieces. Thus, exchanging workpiece receptacles is dispensed with. According to the disclosure, the described method is particularly suitable for welding sheet parts, particularly in the automotive industry. In general, a laser triangulation method, in which previously defined points on the workpiece are measured, is mentioned as a possible measuring system. By way of example, reflectors are to this end applied to the workpiece. According to the description, the position of each reflector can be established by means of a light source and a two-dimensional detector, and so the position and alignment of the workpiece can be captured by using three such points.
What is common to these systems and methods is that the positions of a plurality of labeled points on the object can be established by means of a contactless, photogrammetric coordinate measurement with the aid of image processing systems.
In order to calibrate such a measuring system, with the aid of which an object can be positioned very precisely into an intended position within the scope of an industrial process, it is known to carry out in advance a multiplicity of calibration measurements using a predetermined and routine calibration measurement cycle. Here, the calibration measurement cycle is designed such that it has at least a certain number and variety of calibration measurements. Using the calibration measurement data—captured when the calibration measurements are carried out—it is now possible to determine calibration parameters that at least relate to position and orientation of the recording apparatus of the measuring system in a defined coordinate system, and in particular also relate to internal calibration parameters of individual components of the measuring system such as camera constant, main point, camera distortion, calibration parameters of the angle measuring unit, etc.
In particular, the calibration measurement cycle can have such a number and variety of calibration measurements that the calibration parameters are overdetermined by the calibration measurement data captured in the process. Then, the calibration parameters can be established by e.g. fitting, more particularly according to the least squares method.
By way of example, it is possible as calibration measurements to capture respective images—in different alignments—using the first camera and the respective angular alignments using the angle measuring unit. Moreover, it is possible to determine image coordinates of one or more defined target markers in the captured images using electronic image processing wherein, in particular, the positions of the target markers arranged in space in a defined fashion and/or the distances between the target markers in the coordinate system of the space are known very precisely.
That is to say the calibration measurements in a calibration measurement cycle are for example respectively carried out under variation, in particular a predefined variation, of the angular alignment of the camera or cameras and/or under variation of the positions of the target markers in the coordinate system of the space, with the positions of the target markers and/or a shift of the target markers between the calibration measurements in the coordinate system of the space respectively being known or codetermined. From the overview provided by carrying out such a preprogrammed procedure of a calibration measurement cycle with a multiplicity of calibration measurement data captured during different calibration measurements, it is possible subsequently to establish the current external and internal calibration parameters of the measuring system.
Unstable surroundings result in time-dependent external influences on the measuring system (such as drifts from changes in temperature, vibrations and/or deformations). This may require repeated calibration—particularly of the external orientation parameters of ail recording apparatuses of the measuring system.
There should be complete recalibration of the measuring system, e.g. approximately twice daily, under the usual conditions in order respectively to obtain sufficiently fitted calibration parameters in respect of the current surroundings and external influences. Since—in the case of carrying out a known recalibration measurement cycle according to the prior art—each of these recalibrations requires e.g. approximately 30 minutes or more, this could lead to breaks in production that are a multiple of the production cycle times. In conventional industrial/production processes (e.g. in the automotive industry), usual production cycle times last between approximately 30 and 120 seconds depending on the complexity of the production step/processing process (welding, adhesive bonding, folding, transporting, monitoring, etc.) or of the parts to be processed (doors, body, roof, bonnet, etc.).
This results in the conflicting goals of, firstly, providing a measuring system that was respectively calibrated as recently as possible and hence allows high precision and of, secondly, disturbing/delaying the production advance of the industrial/production process, for which the measuring system is used, as little as possible or, in a best case scenario, not disturbing/delaying it at all. These conflicting goals have until now not been resolved satisfactorily in the prior art.
Moreover, even such “time-based samples” of a recalibration, carried out approximately twice daily, would not permit direct immediate intervention in (correction of) positioning measurements that take place for very precisely positioning the object within the scope of the industrial process.